MicroHS support - #11392

What's achieved

lib:Cabal and cabal-install can build, run, test and install packages with MicroHs, mhs, the way its own package manager, mcabal, does.

AI disclosure: this was written largely by Claude Code (Fable 5.1 or Opus 5.5 depending on the weather outside). I did review it myself and it's a result of many iterations...

Size

The patch looks big but fear not: most of it is contained in the new MHS.hs module and tests.

(Counted with git diff --numstat against master at c433b30.)

Notes:

• The most annoying part and the one to focus your review on is cabal-install. In an ideal world, we'd not need to touch it at all but as the Commit history (below) explains, concessions had to be made.

• Changes to Cabal outside MHS.hs are trivial, which is a strong point.

• Of the lines added to cabal-testsuite, 84 are in the 6 test scripts and 292 in the packages they build.

• 22 of the 28 files in docs are the generated help texts of the commands, which gain --mhs and the mhs program.

Commit history

Each commit ends with how many of the 91 packages of MicroHs's own list build, and run a program, with cabal at that commit, replayed for every commit in the same way, with nothing installed for mhs besides its base (see Testing below and all results in a separate repo).

• cfced1d Adds the mhs program and the backend Distribution.Simple.MHS, to which configure, build, install and register dispatch: it builds libraries into mhs package files and executables against them, compressed with -z as mcabal does, and installs and registers both, in place and in the store; the packages of mhs itself are described with mhs -L. Every extension is accepted, as mhs ignores the pragmas of those it lacks, and so are reexported-modules that keep their names, as mhs sees every module of the packages it loads. 3/91 build and run: with base at 0.16.9.0, the solver rejects most packages from Hackage, or falls back to ancient versions of them, until the next commit.
• cb3743e Presents the base of mhs at version 4.19.1.0, and the packages it bundles, such as bytestring and text, as installed packages at the versions MicroCabal uses, except hashable, at 1.4.7.0, the API that mhs implements, rather than MicroCabal's 1.0.0.0. 39/91 build and run.
• 373d4b4 Passes the macros of cabal_macros.h, such as MIN_VERSION_base, to the preprocessor of mhs as -D options. 58/91 build and run.
• 9338e00 Rewrites, in cabal-install, the packages that the solver sees, as MicroCabal does: ghc-compat, which has the modules that are part of base with GHC, becomes a dependency of every component, and array and random become their forks array-mhs and random-mhs, so that cabal plans and builds all three from Hackage, and mhs needs nothing installed besides base. Rewriting what the solver sees is enough because cabal-install builds packages in-library (#11703): it configures each package with the description in its plan, which is rewritten already, rather than running a Setup script that would read the original .cabal file again. 88/91 build and run.
• b743e6b Puts the C sources of a library, and the headers beside them, beside its package file, where mhs compiles them when it links a program, and gives mhs the include directories of the dependencies with -I. 91/91 build and run.
• f62e6be cabal install --lib installs libraries and their dependencies with mhs -Q into the directory where mcabal installs packages, which is the package directory of mhs itself after make minstall, or into the one given with --package-env, so that plain mhs finds them, rather than writing a GHC environment file. 91/91 build and run.

Testing: 3 parts

Part 1: cabal-testsuite

cabal-testsuite/PackageTests/MHS/Build builds and runs a library with a hidden module and an executable using it, with the bounds of a package from Hackage, and is skipped when mhs is not on the PATH. MHS/Macros checks the version macros and is skipped unless cpphs, which mhs runs for CPP, is on the PATH too. MHS/CSources checks the C sources of a library built in place and of one installed in the store from a local repository, each with a private header beside them, and MHS/InstallLib installs a library with cabal install --lib and compiles a program against it with mhs alone. MHS/Preprocess builds a library module and the main module of an executable from .hsc files, which hsc2hs turns into Haskell in the build directories, with a C function the library defines with #def, and is skipped without hsc2hs; MHS/ReexportRenamed checks that a library re-exporting a module under another name, which mhs cannot provide, is rejected. The validate workflow installs MicroHs from source on Linux for the GHC of the release, in about a minute on a cache miss, so that the tests run in CI; elsewhere they are skipped. To run them locally:

cabal run cabal-testsuite:cabal-tests -- --with-cabal="$(cabal list-bin cabal)" cabal-testsuite/PackageTests/MHS/Build/cabal.test.hs cabal-testsuite/PackageTests/MHS/Macros/cabal.test.hs cabal-testsuite/PackageTests/MHS/CSources/cabal.test.hs cabal-testsuite/PackageTests/MHS/InstallLib/cabal.test.hs cabal-testsuite/PackageTests/MHS/Preprocess/cabal.test.hs cabal-testsuite/PackageTests/MHS/ReexportRenamed/cabal.test.hs

Part 2: MicroHs's own package list

MicroHs's Makefile.packages is the list of 91 Hackage packages its hackage-ci workflow installs with mcabal on every push. https://github.com/ulysses4ever/cabal-mhs-testsuite builds them with cabal and mhs instead, one at a time and in the same order, each into a store shared by all the steps, with this PR's commits applied; its CI also runs cabal-testsuite/PackageTests/MHS. Every step also runs a small executable depending on the package. As mhs compiles C sources when it links a program, even the executable that does nothing compiles the C sources of the package and its dependencies. For time, splitmix, random, QuickCheck, hspec, terminfo and binary, the executable calls into the package: QuickCheck passes 100 tests, hspec reports 1 example, 0 failures, and terminfo links with the curses library its configure script found. Where mcabal differs from cabal, the replay does what mcabal does, and its README says how: the --git overrides become source-repository-package stanzas, and since mcabal ignores version bounds, a few are lifted in project.extra, each with its reason. mhs has nothing installed besides base; cabal builds ghc-compat and the forks itself.

All the counts in the list of commits are for compiling each package as it is published, with no settings of its own: the project file sets no flags and no compiler options for any package of the list. It only chooses versions and sources: the index-state, the git repositories of MicroHs's Makefile, the three bounds that project.extra lifts and its one constraint, and a copy of hspec-core with its missing dependency added. The tiny executable of each step, which is not one of the packages, is linked with --no-as-needed on Linux.

The version macros take the count from 39 to 58: without them, transformers, which defines its Semigroup instances under #if MIN_VERSION_base(4,9,0), blocked 26 packages, and prettyprinter stopped at its own #error "MIN_VERSION_base macro not defined!". The rewrite takes it from 58 to 88: array from Hackage, which mhs cannot compile, blocked 17 of those 30 packages until array-mhs replaced it, and without ghc-compat, exceptions and call-stack failed on GHC.Stack, and time on Language.Haskell.TH.Syntax. The last three, time, binary and these, through binary, needed the headers of C sources, such as HsTime.h, which the commit for C sources puts on the include path.

One package needs a fix to build with cabal: hspec-core imports async without depending on it, as it compiles its vendored copy of async only for GHC. That works with mcabal because mhs sees every package that mcabal has installed. Cabal gives mhs only the declared dependencies, so the replay adds if impl(mhs) / build-depends: async to a copy of hspec-core, which is the fix that belongs upstream; without it, hspec-core and hspec above it fail.

Running the programs found problems outside cabal, in mhs and in some packages, which mcabal has too; INVENTORY.md lists them with reproductions.

Part 3: Building MicroHs itself (after patching it up)

MicroHs builds its base with mcabal build in lib. We allow cabal build it with mhs too, and the compiler from MicroHs.cabal, once four problems of those .cabal files that mcabal does not mind are fixed:

1. library base has to be library,
2. Data.Float is listed twice,
3. six listed modules of base have no source,
4. MicroHs.cabal lists for every compiler seven modules that only GHC needs.

microhs-cabal-fixes.patch fixes them, and MicroHs still builds with GHC with it. base also needs allow-boot-library-installs: True, as with any boot library.

The base that cabal builds has the same modules, exports and definitions as the one mcabal builds. cabal is no slower than the Makefile and mcabal, and, as it compresses programs as mcabal does, it builds an mhs of exactly the size of the Makefile's: the same machine code, and the same combinator code but for how 52 source locations of base are spelled. The testsuite's README has the times and sizes. An installation of mhs whose base and compiler cabal built passes the tests of cabal-testsuite/PackageTests/MHS and builds and runs the whole package list, and the testsuite's CI builds both on every run (CI run). cabal install exe:mhs does not give a working mhs, though: mhs looks for its packages and runtime only beside its binary, in the layout of its own installer.

INVENTORY.md lists every issue or wrinkle this work found outside cabal: in mhs, the MicroHs repository, MicroCabal, and some packages on Hackage, with reproductions.

Not covered yet

• mhs-options in .cabal files, cabal repl, Backpack, foreign libraries, profiling and shared ways.
• The replay runs a small program per package, but not the test suites of the packages. Running those with mhs is the next step there.
• Upstream: hspec-core should depend on async for mhs, mhs could publish the versions at which its bundled packages should be presented, mhs could pass the libraries of packages after the C files, declare the C functions a program imports and expand $VERSION in mhs.conf to its version; MicroCabal could move to time 1.16.0.1, which fixed the time zone that time 1.15 gets wrong under mhs.

Checklist

• Patches conform to the coding conventions.
• Any changes that could be relevant to users have been recorded in the changelog.
  • Is the change significant? If so, remember to add significance: significant in the changelog file.
• The documentation has been updated, if necessary.
• Manual QA notes have been included.
• Tests have been added. (Ask for help if you don’t know how to write them! Ask for an exemption if tests are too complex for too little coverage!)

QA Notes

A smoke test, and how to extend it in the direction you want to check.

Set up. Install MicroHs v0.16.9.0 from source, which also installs cpphs, the preprocessor that mhs runs for CPP and that the nixpkgs package lacks, and build cabal from this branch:

git clone --branch v0.16.9.0 https://github.com/augustss/MicroHs
make -C MicroHs -j all && make -C MicroHs minstall   # installs into ~/.mcabal; add ~/.mcabal/bin to PATH
cabal build cabal-install:exe:cabal
alias cabal-mhs="$(cabal list-bin cabal)"

Smoke test. A library, and an executable using it:

mkdir -p mhs-demo/src mhs-demo/app && cd mhs-demo
printf 'packages: .\ncompiler: mhs\nwith-compiler: mhs\n' > cabal.project
cat > mhs-demo.cabal <<'EOF'
cabal-version: 3.0
name:          mhs-demo
version:       0.1.0.0

library
  exposed-modules:  MyLib
  build-depends:    base ^>=4.19, bytestring
  hs-source-dirs:   src
  default-language: Haskell2010

executable mhs-demo
  main-is:          Main.hs
  build-depends:    base, mhs-demo
  hs-source-dirs:   app
  default-language: Haskell2010
EOF
cat > src/MyLib.hs <<'EOF'
module MyLib (greeting) where

import qualified Data.ByteString.Char8 as B

greeting :: String
greeting = B.unpack (B.pack "hello from mhs")
EOF
cat > app/Main.hs <<'EOF'
module Main (main) where

import MyLib (greeting)

main :: IO ()
main = putStrLn greeting
EOF
cabal-mhs run mhs-demo   # prints: hello from mhs

Expected: Build profile: -w mhs-0.16.9.0 -O1, and ghc-compat built from Hackage first, as every package depends on it with mhs; base ^>=4.19 and bytestring resolve to what the base of mhs provides. With -v2, the mhs commands show the library compiled into a package file, and the executable compiled with that file and base preloaded (-p) and the search path of mhs cleared (-a).

Extend it. The tests in cabal-testsuite/PackageTests/MHS do most of these:

• A dependency from Hackage: add random to the library's build-depends, and cabal-mhs build builds the fork random-mhs instead.
• CPP: add {-# LANGUAGE CPP #-} to MyLib, and choose code with #if MIN_VERSION_base(4,19,0) (MHS/Macros).
• C sources: give the library c-sources: cbits/twice.c, include-dirs: include and install-includes: twice.h, and MyLib a foreign import ccall "twice.h twice" twice :: CInt -> CInt; mhs compiles the C file when it links the executable (MHS/CSources).
• The store: cabal-mhs install exe:mhs-demo builds the package in the store and installs the executable.
• Plain mhs: cabal-mhs install --lib installs the library where mcabal installs packages, into ~/.mcabal/mhs-0.16.9.0, which make minstall made the package directory of mhs itself, so that mhs Prog.hs compiles a program that imports MyLib (MHS/InstallLib).

🤖 Generated with Claude Code

https://claude.ai/code/session_0176vAefrw8jPKXdh9h1wKuj